Parameter Tuning by Pairwise Preferences

Most computer vision algorithms have parameters. This is a fact of life which is familiar to any researcher in the field. Unfortunately, for algorithms to work properly, the parameters have to be tuned. We propose a semi-automatic approach to parameter tuning, which is general-purpose and can be used for a wide variety of computer vision algorithms. The basic setup is as follows. The vision algorithm takes as input (i) an actual input (commonly an image) and (ii) parameter values. From the input and the parameter values, it produces an output (sometimes an image, sometimes another quantity). Thus, a single run of the vision algorithm may be characterized by the triple (input, parameter,out put). The vision algorithm is run several times, leading to several such triples. The user is then given pairs of outputs, and asked to judge which output is preferred, in that it constitutes a higher quality output. The user provides such a pairwise preference for several pairs of outputs. Based on these pairwise preferences, the goal is to find a function over the parameter space which respects the user’s preferences, i.e. such that if parameter1 parameter2 then the function is larger for parameter1 than for parameter2. Let the parameters be denoted by x. Then this problem can be posed as the following optimization over functions f :